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Abstract: One of the means of transforming classroom experience is by conducting 
action research with students. This paper reports about the action research with 
university students. It has been carried out within a semester of the course “Methods 
of Upbringing”. Its goal has been to improve evaluation of higher education teaching. 

Different forms of summative and formative evaluation have been devised 
with an emphasis on creating critical friendship between student and professor 
(authors of this article). Video recordings of lessons have been utilized for realization 
of critical friendship. At the end of lesson students have been offered various 
questionnaires and evaluation sheets, and an open standardized interview has been 
conducted with a group of students. A workshop dedicated to evaluation of higher 
education teaching has been carried out as well. 

This research has shown that students can actively participate in evaluation, 
and that their comments and suggestions should stimulate teachers to improve all 
stages of teaching process, including evaluation. The authors believe that evaluation 
of higher education teaching could be brought to a higher level by educating teachers 
and students about the importance of evaluation for the quality of teaching process 
and finding ways to include students in this process. 

Keywords: evaluation, fourth generation evaluation, higher education teaching, 
critical friendship, action research. 


1. Introduction 

Evaluation of teaching practices needs to be a part of every teaching 
process. However, this was not the case 40 years ago when the first book on 
evaluation of higher education appeared in Great Britain. Hounsell (2003) 
reported that this topic, at that time rather controversial, was surprising, scary 
and insulting for many academics. Similar feelings were evident in Croatia 
after the introduction of the Bologna Process, which emphasizes the necessity 
to evaluate accredited programmes of higher education institutions in order to 
ensure sustainable quality in higher education. 
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According to Patton (1990, pp. 11-12), evaluation as a type of action 
research aims at providing information about someone’s work and improving 
it, dealing with individual and social problems. So, the evaluation of 
instruction practices is primarily aimed at the development of teaching process, 
along with teacher competences. A good teacher is the one who examines their 
instruction practices, develops their teacher competences, and evaluates 
teaching process in accord with its influence on learners (Brown et al., 2003). 

Feedback about teaching practices may be obtained from different 
sources. According to Hounsell (2003), data may be collected through 
evaluation done by students, colleagues or associates, and from self-evaluation 
data. Useful indicators may be: course attendance, exam success, number of 
students enrolled, grading criteria, students’ fatigue or activity in classes, etc. 
Student evaluation of teaching practices may be conducted by means of 
evaluation sheets, questionnaires, group discussions or electronic 
questionnaires. 

Evaluation may be summative or formative. Summative evaluation, 
according to Patton (2002, p. 218), investigates the overall efficiency of a 
programme, policy or a product, in order to decide on its sustainability and 
further implications. Summative evaluation is rarely based on qualitative 
research data, but qualitative approach gives summative evaluation certain 
depth and refines quantitative data. 

Formative evaluation means that the realization of teaching is 
continuously assessed in order to make it richer and better. Everybody who is a 
part of educational process (in any way) may take part in formative evaluation, 
but most often teachers and students do it. Formative evaluation relies on 
different type of data, qualitative indicators being especially important (Patton, 
2002). Scriven (1966) has shown that both kinds of evaluation are equally 
important in research about education. 

Guba and Lincoln (1989) have distinguished among four generations of 
evaluation. First generation evaluation uses tests to measure students’ 
achievements. This approach is based on the belief that education is supposed 
to teach students commonly accepted truths and general knowledge and that 
students are expected to demonstrate their knowledge answering questions in 
different exam situations. Based on that interpretation of evaluation, numerous 
standardized tests have been designed. The most famous is the intelligence test. 
In Croatia, the state graduation exam project is an example of the first 
generation evaluation. 

Second generation evaluation puts emphasis on the description of 
teaching curriculum advantages and disadvantages related to the established 
educational goals. This type of evaluation is similar to formative evaluation, 
except that the results are visible only after the programme has been 
completed. Based on the analysis of collected data, curriculum is to be 
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improved and changed until expected results are achieved. Particular 
instruments developed during the first generation evaluation represent only a 
certain number of actions to be used during evaluation process. The system of 
internal education quality assurance at faculties is designed this way. 

Second generation evaluation is basically descriptive; therefore there 
has been a need for a type of evaluation that would involve judgement. That is 
the main feature of third generation evaluation. Evaluators take over the role 
of judges while continuing to use evaluation procedures from previous 
generations. High quality judgement requires a high level of proficiency. 
Therefore evaluation has been reassigned to experts who do not only evaluate 
performance, but goals as well. An example of this type of evaluation is the 
external independent periodical assessment of internal education quality 
assurance at higher education institutions, conducted by Education and Teacher 
Training Agency in Croatia. 

Although every new generation of evaluation has been more advanced 
than the previous, Guba and Lincoln (1989) have concluded that all three 
generations have certain disadvantages which are evident in the fact that the 
level of democracy is low. Therefore they have suggested the implementation 
of fourth generation evaluation with the following ideas behind it: Evaluation 
is a socio-political process influenced by social, cultural and political factors. 
Evaluation is a cooperative process which implies consulting and the right of 
all interested parties to express their opinion. In that process all persons 
involved are learners and teachers at the same time. It is a continuous, 
recursive and divergent process that does not encompass any eternal truths. 
Conclusions that result from the process of evaluation may be disproved by 
new information or by introducing more sophisticated evaluation procedures. It 
is not possible to plan evaluation in detail, because every step of the process 
depends on the results of the previous step. Besides, it is a process with 
invisible results. Evaluation is a creative process in which truth is not found, 
but created. This action research has been carried out to a great extent using 
principles of fourth generation evaluation. 

We believe that it is especially important to improve and sustain the 
quality of studies for future teachers and educators. Higher education courses 
ought to be models for future teachers and educators according to which their 
own competences will be developed. Quality evaluation is very important for 
the quality of higher education or any other level of education. Unfortunately, 
it is still quite often conducted in a non-systematic and superficial manner. 
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2. Research 

2.1. Choice of research approach 

As the intention of this research has been to improve evaluation of our 
practical work, action research has been chosen as the research approach, since 
it is directed at changes, not simply at theoretical explanations of certain 
phenomena (Carr & Kemmis, 1986; McNiff & Whitehead, 2010; Reason & 
Bradbury, 2006). Besides, this research approach is the closest to fourth 
generation evaluation used in this research (Guba & Lincoln, 1989; Heron & 
Reason, 1997, p. 284). According to the authors of this chapter, action 
research is a systematic, creative and cooperative action based on: 
philosophical consideration of values, creative design of new procedures aimed 
at achieving essential changes, collection of data on the changing process, 
(self) critical analysis of results, and looking for ways to implement action 
research experiences in the culture of close and wide community. 

Action research has been increasingly used in the last 20 years in 
Croatian pre-school, primary school and high school institutions, but not in 
higher education. In other countries of the world, action research is often used 
to improve higher education and to develop theoretical knowledge (Atweh et 
al., 1998; Norton 2009). We hope that other academics will be encouraged by 
this research to use action research more often. 

2.2. Action research context 

This action research has been done by the 3 rd year student of philosophy 
and pedagogy Maja Bungic and Assistant Professor Branko Bognar as a part of 
the 2010/2011 summer semester course Methods of Upbringing. Teaching 
assistant Ruzica Pazin Ilakovac participated as a teacher and critical friend. 
The research was done with 37 first-year-students of Pedagogy studies. Classes 
took place on Fridays from 8:00 till 11:15 in the so called “pedagogy 
classroom” at the Faculty of Philosophy in Osijek. This classroom’s 
arrangement (Picture 1) is quite different from that of other classrooms at the 
faculty, which are mostly suited for the traditional teacher-fronted - instruction. 
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Picture I. Pedagogy classroom 

2.3. Action research problem 

We believe that university teaching ought to consider students’ needs 
and interests and provide active and creative participation at all stages of the 
teaching process: consulting, realization and evaluation (Bognar & Matijevic, 
2002). Unfortunately, such practices are not common. Students claim that there 
is no productive interaction and cooperation during classes; instead a student is 
just a passive observer of a teacher’s activity. Students sometimes feel that no 
one listens to them when they talk about problems that quite often appear in 
classes. 

To change this situation, a quality and democratically constructed 
evaluation might be useful. It would allow students to openly discuss 
possibilities of teaching improvement with their teachers. Unfortunately, at the 
time when this research was done, course evaluation at our higher education 
institution was conducted through an anonymous student questionnaire at the 
end of each semester. This type of evaluation is not satisfying considering 
students’ and teachers’ needs, nor does it contribute to the improvement of 
teaching. We believe that academics who teach at teacher training studies 
should not only enhance the quality of their teaching, but the quality of 
teaching evaluation as well. Faced with the problem of insufficient activity of 
students in classes, especially in regard to the criteria for teaching evaluation, 
evaluation design and realization, we decided to establish cooperation in order 
to improve the evaluation of teaching in higher education course Methods of 
Upbringing. 
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2.4. Action research plan 

First, an action research plan was designed. The following research 
goals were agreed on: (1) to design and carry out different forms of formative 
and summative evaluation (2) to involve students in the design of evaluation 
procedures (3) to take account of students' satisfaction with teaching and 
especially with teaching evaluation. 

The following criteria were set for the subsequent evaluation of research 
goals: (1) Formative evaluation was conducted using various evaluation 
procedures; formative evaluation made it possible for all the participants of 
teaching process to help improve its quality. (2) Summative evaluation has 
been conducted at the end of lesson using various evaluation procedures. (3) 
As the ideas of fourth generation evaluation suggest, it was important to 
construct evaluation forms which would democratically include all the 
participants of teaching process, above all students. (4) Evaluation results 
indicate satisfaction of students with the quality of teaching in the course 
Methods of Upbringing 

Activities aimed at achieving research goals were grouped according to 
evaluation type: 

a) Formative evaluation that involved critical but friendly discussions that 
were to be held after classes and later by e-mail with the intention of discussing 
our impressions and possibilities of improvement of teaching practices in 
Methods of Upbringing. Besides, feedback was to be received about classes. 
Workshops were to be video recorded and photographed to be analysed and 
commented later on. Questionnaires and evaluation sheets were to be given to 
students at the end of each workshop. An open thematic interview was to be 
conducted with a group of students. 

b) Summative evaluation was planned to be conducted as a part of the 
workshop at the end of semester, which would deal with evaluation of higher 
education. Its intention was to find out how evaluation was conducted, how 
satisfied students were with its quality, and how they could contribute to its 
improvement. Also, students were to fill in the final evaluation sheet for the 
course Methods of Upbringing. 

Different forms of formative evaluation were to be conducted during the 
whole semester. Summative evaluation was to be conducted during 4 classes 
( 1 80 minutes). 

2.5. Action research implementation 

At the beginning of this research it was agreed that student Maja Bungic 
would prepare different evaluation forms after every workshop. Different 


144 




Branko Bognar, Maja Bungic: Evaluation in Higher Education 
Zivot i skola, br. 31 (1/2014.) god. 60., str. 139. - 159. 

evaluation forms, video recordings and photographs of workshops were a part 
of formative evaluation of teaching. 

2.5.1. Evaluation of the workshop “What is a man?” 

At the beginning of the workshop Assistant Professor Branko Bognar 
consulted students on class schedule. Also, students were divided into 6 
groups, which was a prerequisite for cooperative learning. Each group 
presented their expectations concerning the course, colleagues and teachers, 
and decided on the name of the team, its features and motto. Then, the teacher 
presented the course curriculum and student Maja Bungic as his critical friend. 
Together, we presented our research plan and asked students for a written 
permission to video record and photograph classes. They gave us one. Then the 
activity “Alien encounter” followed, whose goal was to do a role-play which 
would introduce aliens to human species who live on Earth. Then, students 
were given a task to think about it and discuss why Neanderthals died out, 
whereas our species survived. Students also watched a movie about 
Neanderthals. Then they read some chapters of relevant literature (Hegel, 
1966; Cassirer, 1978; McBrearty & Brooks, 2000) and answered questions, 
participated in a discussion and created a comic on the similarities and 
differences among humans (homo sapiens) and other beings, especially the 
extinct species of hominid - Neanderthals. A conclusion was reached that 
humans were self-aware, free, creative beings who create their own world. 

During her first visit, student Maja watched and recorded the class (see 
http://bit.lv/HoTdJm ~). She noticed certain issues concerning activity duration, 
which she elaborated in her research diary: “Certain activities might have 
taken up less time (e.g. “Alien encounter"), so that there is more time for the 
presentation of comics Differences between human and Neanderthal education 
at the end ” (M. Bungic, personal communication, March 7, 2011). In our 
conversation and e-mail correspondence, duration of the activities of the first 
workshop was emphasised as an important problem: 

When organizing workshops, a problem which usually appears is the 
problem of timing. On the one hand, participants need to have enough time to 
think an activity through and carry it out, and on the other hand it is important 
to finalize and comment on the planned activities. Besides, different activities 
have different importance. (B. Bognar, personal communication, March 7, 
2011) 

This problem was partially caused by technical and organisational 
difficulties at the beginning of the class, which were a result of looking for an 
available classroom 1 , preparing technical equipment, waiting for students who 
had been late, etc. 


A problem appeared due to some timetable issues - their lecture coincided with some other 
obligations. 
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The class lasted 10 minutes longer because of that, so students became 
restless. It affected the final evaluation by means of an evaluation sheet I 
prepared. I am afraid that students completed it in a hurry without deep 
consideration of what went on during class (Research diary, March 7, 2011). 

Outcomes of evaluation therefore may be influenced by various factors, 
such as organisational conditions, activity duration, lesson duration, lack of 
concentration, fatigue, etc. Experiences that precede evaluation may affect 
evaluation approach. We believe that the effect of “last experience” could be 
reduced by giving students a chance to evaluate activities several times during 
classes, especially if classes are long enough, as was the case with our lesson. 

In the following 90-minute lesson there was no problem with activity 
duration, which is obvious from the video recording of critical, but friendly 
conversation (4:55). The conversation took place after the lesson 
( http://vimeo.com/2665 1463 1: “Duration of activities was optimized today and 
there were 10 minutes left for evaluation. There was no pressure, so... they 
were able to think about what they would respond ” (M. Bungic, personal 
communication, March 1 1, 201 1). 
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Picture 2. Evaluation sheet “Neanderthals “ 


Evaluation sheet for students at the end of the workshop contained a 
drawing of a Neanderthal family (Picture 2) and students needed to write down 
what they liked or did not like about the class, from the perspective of one of 
the family members. Also, they needed to write down suggestions on how to 
improve classes in the future. Results revealed that about half of students liked 
all the activities, whereas 37.5% of students highlighted the movie about 
Neanderthals: “Today I enjoyed cooperative learning and watching the movie”. 
Students also liked cooperative learning, creativity and class dynamics. They 
did not like the fact that some of their colleague students did not behave 
politely during the class, that they had little time for practical activities and that 
the class lasted longer than usually. One student stated: “I did not like it that 
not all students were interested”. A part of students (19%) suggested that all 
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classes should be like that day’s class, whereas 12.5% of students suggested a 
more realistic duration of activities. 

We noticed that students highlighted things that they liked about the 
class, whereas they rarely pointed out negative aspects of the class or gave 
suggestions for teaching improvement. This might mean that they were 
satisfied with teaching practices. As it was important for us to hear students’ 
comments on both positive and negative aspects of teaching, and give 
suggestions for the improvement of teaching, we were not quite satisfied with 
the evaluation results. We think that it would be possible to stimulate students 
to give more detailed feedback by explaining evaluation procedure and 
discussing its importance. 

2.5.2. Critical account of the video of the lesson 

We commented on the video recordings of workshops. The teacher 
would prepare digital recordings which would be watched by both, the student 
and the teacher. The student would then write down her remarks. Most of the 
recordings and comments were sent to each other by e-mail, which made the 
communication and cooperation much easier. Here are the student’s 
suggestions regarding the way some activities were carried out: 

In the introduction students could have been stimulated to think 
about the first question that was asked - “ What does a man have 
to do with this course Methods of Upbringing and why do we 
mention a man in this context at the beginning of this course? ” 

Only one student expressed his opinion. I think that others 
should have been stimulated to take part in the discussion. That 
could have been done by an activity in which students would 
link the concepts humans and social education and reach some 
personal conclusions about the course; or by an activity that 
would reveal what makes us human; or how social education 
affects human beings (M. Bungic, personal communication, 

April 16,2011). 

The teacher accepted and further developed the student’s suggestions: 

I agree with you, Maja. The question I asked is very important 
and perhaps more time should have been spent dealing with it. I 
will by all means try to predict some other possible activities 
related to that question. Here is an idea: Students sit in the circle. 

The teacher reads a statement about humans (e.g. "A man is a 
creative being”) and throws a ball to one of the students. The 
student is then supposed to say something about social education 
inspired by the statement he or she has heard from the teacher. 

Then this student throws the ball to another student who again 
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says a sentence beginning with “A man is... ” What do you think 
about that idea? (B. Bognar, personal communication, April 17, 

2011 ). 

Except for the problems mentioned above, the student suggested some 
other solutions or revealed personal impressions about certain activities. 
Although students are sometimes not fond of such activities, 1 
personally like it when students have the opportunity to role-play, 
because it may raise their spirits and they may become more 
actively engaged in agreeing on the roles and scenario. Also, 
acting can help them conquer stage fright, which is a good 
starting point for their future job (M. Bungic, personal 
communication, April 16, 201 1). 

Commenting video recordings was useful for it helped us see problems 
from a different angle and hear some new ideas and solutions. After repeated 
viewing of video recordings we were able to spot some details that might have 
been a cause of some problems. They would have been very difficult to 
perceive in a regular classroom situation. During our joint commentary on 
video recordings, we started to understand the value of this kind of cooperation 
between critical friends. In one of his e-mail messages to his student, Professor 
B. Bognar said: 

“Maja, the way we comment on teaching practices is what 
cooperation of teachers and school counsellors is all about. I 
believe that our cooperation model will be helpful in your future 
job as a school counsellor” (personal communication, April 17, 

2011 ). 

It is to be concluded that during the process of evaluation, students may 
contribute by giving comments from a perspective different than teacher’s. 
They can stimulate a teacher to see his/her teaching practices from a different 
angle, and try to improve them, which is stated in the following teacher’s 
words: 

“Your comments and suggestions have made me think and have 
helped me see some important problems. Also, you have given me 
some good pieces of advice which could improve the quality of 
my teaching” (B. Bognar, personal communication, April 17, 

2011). 

It is possible to include other individuals besides action research 
participants in the critical and friendly discussion, even if they live outside 
Croatia. We prepared English subtitles 2 for YouTube videos (see 


2 YouTube and Vimeo public services were used for video sharing 
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http://bit.lv/HRHFU and http://bit.lv/HplWfg ). Besides, we prepared an 
English version of the complete report on the activities done so far, with the 
results of evaluation included (see http://bit.lv/IfDU 1 x l. Discussion took place 
at a network cooperation system ( http:/7pest-prog.ning.com l and at workshops 
for training educational experts “Pestalozzi core knowledge, skills and attitudes 
for all teachers 4 *, organized by the Council of Europe. 

Two experts joined the discussion - one of the leaders of Pestalozzi 
Programme and Jack Whitehead, who advocates life-theoretical approach to 
action research. The leader of Pestalozzi Programme referred to the matter of 
emphasising positive features of humans, and not discussing negative aspects 
of human behaviour. She suggested that one should discuss it with students 
why some people become destructive. In her opinion, one cannot heal a disease 
if one is not aware of its causes. After these critical comments a discussion 
developed about various topics such as baseline values and some ideas on how 
to improve workshops. Both participants in the discussion said that they had 
been inspired to think, write and read about the topic. 

Jack Whitehead commented on video recordings through the network 
cooperation system for Pestalozzi project members like this: 

Dear Branko, the link to your report was available and I 
sincerely enjoyed reading it and watching video recordings. I am 
aware that you would like to be given some answers which might 
help you improve your teaching practices. The feeling that / got 
based on the report data is that you have already been given 
great critical evaluation from your students and critical friends, 
who have helped you to improve your teaching practices. / like it 
how open your evaluation conduct was. I also like the values that 
have been directed towards overcoming/understanding 
discrimination and emancipation issues i { J. Whitehead, personal 
communication, April 21, 201 1). 

After this, an almost completely positive comment, the discussion did 
not continue. It is to be noticed that critical comments made the teacher think, 
discuss and read related literature, whereas positive comments, besides being 
good for his professional confidence, did not influence his deliberation, 
teaching practice improvement, or his need for professional learning to any 
great extent. We think that, although positive comments that build up 
professional confidence are necessary for critical friendship, honest critical 
comments are as important because they can initiate changes (Handal 1 999). It 
is recommended that critical friends use both approaches, although that cannot 
be expected from all people. Our example shows that one critical friend 


The free software DivXLand Media Subtitler was used to put subtitles on the videos 

( http; w ww.divxland.ore'Subtitler.nho ) 
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(Whitehead) emphasized positive aspects of teaching practices, whereas other 
comments were more inclined to criticism. 

In any case, we may agree with Elliot (1993, p. 176) when he said that 
teachers can improve their teaching practices to a great extent by simply 
communicating with other people, especially other professionals. An important 
role in that process belongs to learning communities (Stoll & Fink, 2000), 
which may consist of people from the same professional circles (e.g. 
colleagues from the same faculty or department), or distributed communities of 
practice (Wenger et al., 2002) within a national or international project 
(Pestalozzi project in our case). In the case of the latter, the latest network 
cooperation systems are an important factor in establishing communication. 

2.5.3. Interview with students 

Three workshops had been observed and filmed and after that an 
interview was conducted. Five students were interviewed in professor Bognar’ s 
office, which was video recorded (see http://vimeo.com/26647779 ). The 
interview was conducted by student M. Bungic. The teacher was not present. 
Students were asked about their impressions about teaching practices in the 
course Methods of Upbringing. Their answers implied that they mostly liked 
the course: 

“/ like the way this course was taught, because it was creative 
and supportive. ", 

"I like the fact that professor Bognar was enthusiastic and that 
he gave us hope that some things could be changed in our 
education system ”. 

Students were asked to compare teaching practices in this course with 
teaching practices in other courses and all of them reported noticeable 
differences. They stated that in most courses students were just passive 
observers in the class, whereas professor was the one who would lead and 
decide. A student said: “I don’t like to sit and watch a professor talking, and 
then when you ask a question, he looks at you as if thinking ‘And why have 
you asked that question?’. I like it when there is some interaction among 
people, when people talk to each other, when some conclusions are reached”. 
Another student said: “In this class there are no wrong answers. Whatever you 
say, your opinion is valued.” When asked what they thought of Professor 
Bognar’s teaching, one student said: “I like professor’s energy which is rare to 
find. Others find it much easier to sit down and “click” a presentation.” 

If they had a possibility to teach this course, all the students would 
choose creative approach and cooperative learning techniques. One of the 
students said: “I think that I would organise classes in a similar way. I would 
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search for new ideas, I would prepare well, and I would do a little research. I 
might organise some field work, too.” 

According to Patton (1990, p. 335-337) this interview that M. Bungic 
conducted is so called focus group interview. It was developed in 1950s as a 
part of a market research to get precise feedback from users. Interviewees need 
to be members of a homogenous group of people who answer questions, but 
they also need to listen to other interviewees’ answers. 

Due to the fact that the interview was recorded by a video camera, there 
was concern that students would not be open and that they would feel nervous. 
However, we did not notice any negative influence of its presence. Students 
opened up and answered questions about the course Methods of Upbringing, as 
well as some other courses honestly and (self) critically. Focus group interview 
turned out to be very effective, because students could complete each other’s 
answers. Besides, they felt the group support which probably decreased the 
number of anticipated answers. The fact that a student conducted and recorded 
the interview is also believed to be a positive influence on students’ answers, 
because they did not feel as if they were examined. It felt like a pleasant 
conversation among colleague students instead. Patton (p. 336) points out that 
one of the disadvantages of this type of interview is the interviewing 
experience it requires. In our case, the student M. Bungic seemed to be up to 
the task and conducted the interview successfully, making sure that all 
participants took part equally. 

Using an interview to evaluate teaching practices is very efficient for 
receiving feedback from students or other teaching process participants. 
Interviews with students enable researchers to receive information which 
would be difficult to gather by other evaluation procedures, due to quite short 
duration of class activities. 4 . A recording enables them to see details which are 
sometimes not possible to be said. These data can be important during 
qualitative analysis. 

2.5.4. Evaluation sheets 

Subsequent to the three workshops, students were given evaluation 
sheets by Professor B. Bognar. The evaluation sheet consisted of 15 statements 
related to students’ satisfaction with activities, their management, interaction, 
and workshop participants’ cooperation. Evaluation was based on grading 
these statements using grades that ranged from 1 (“I completely disagree”) to 5 
(“I completely agree”), or 0 for “I cannot reply”. Statements referred to: the 
extent to which students’ educational needs were met, the activity pace, the 
relevance and usefulness of materials and resources, the creativeness 
opportunities for students, the relevance of the workshop for students’ future 


4 Our experiences and considerations related to the use of interviews are available online in a video 
recorded immediately after the interview fhtti vimeo.com/26652626 '). 
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jobs, etc. Also, there was a space provided for individual descriptions of 
everything students liked or disliked about the workshops, as well as their 
suggestions for the improvement of teaching practices. 

Questionnaire results revealed that students were least satisfied with 
how the contents met their educational needs (3.83 out of 5). Possibility to 
express creatively got the highest grade (4.65 out of 5), the level of interaction 
(4.52) and student cooperation (4.55) followed. So, students once again 
confirmed that the course Methods of Upbringing was an opportunity for 
creative and cooperative learning. This information was obtained in the 
interview and during activities of the workshop “Possibilities of evaluation in 
higher education” where students had defined criteria of quality higher 
education. 

Every year at our faculty students fill in the so called student 
questionnaire, an online questionnaire, which assesses students’ opinions of 
the quality of courses and teachers. In the workshop “Possibilities of evaluation 
in higher education” during the discussion, some issues related to the student 
questionnaire were raised. One of the students said: “Names of professors are 
mixed up, and professors get angry when they receive the questionnaire 
results”. One student noticed: “We are all concerned whether it is really 
anonymous; therefore students do not dare take the survey. Also, we are not 
informed enough. They just come and say: “At that time, there!” Students 
believe that student questionnaire is just a formality which needs to be done 
and that its real purpose is not teaching process improvement. The main cause 
why students do not have a positive questionnaire experience is, as they say it, 
the lack of information on its aims. 

Also, they think that it is important to design the questionnaire well, 
because inadequate questions may lead to loss of motivation on students’ 
behalf. It is very important to inform professors on questionnaire aims. Some 
students say that professors take this type of evaluation personally and feel 
insulted if their grade is low. 

Despite all the above-mentioned problems, evaluation sheets may be a 
valuable instrument for data collection if students have been properly 
motivated for taking a part in evaluation and if they believe that the aims will 
really be met, in other words that they would be in position to influence 
teaching improvement. 

It is advisable to make sure that students fill in the questionnaire 
anonymously, to ensure their honesty. Evaluation sheet may be used during 
summative and formative evaluation of teaching practices. We think that 
students might design their questionnaires, conduct them, review, present and 
interpret the results. Besides, evaluation sheets results may be an impetus for a 
discussion about the possibilities of teaching improvement. 
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2.5.5. Workshop “Possibilities of evaluation in higher education” 

At the end of summer semester 2011 student M. Bungic conducted a 
workshop “Possibilities of evaluation in higher education” which dealt with 
summative type of evaluation. The aim of the workshop was to introduce 
students to the possibilities of evaluation in higher education, to present 
different types of evaluation procedures, to involve them in the evaluation 
process and then evaluate the course Methods of Upbringing. The workshop 
began with a miming task during which students needed to show how they 
were feeling at that time and how they would like to feel at the end of the 
workshop. It was an opportunity for students to bring up the issue of the early 
beginning of class once again, which they did not find suitable and which had 
been a problem for them the whole semester 5 . 

After an energizing activity, there was an activity which consisted of 
defining criteria for good quality higher education and discussing its definition. 
Students agreed that quality teaching practices imply a competent and expert 
teacher, as well as active involvement of students through cooperative and 
creative approach. A representative of one of the groups said: “The criteria that 
we find important are: student involvement, understanding of each others’ 
needs, appreciating other people’s opinions, creative activities, professor’s 
competence and expertise, variety of contents and good cooperation”. We have 
noticed that students are aware of the fact that teachers very often use teacher- 
centred methods in their classes, and that active involvement of students in 
teaching process is very rare. Students however recognize the importance of 
their involvement in the design of the teaching process and that is very often 
emphasized as the criteria for good quality teaching. 

The next activity was a circular discussion about the ways of evaluating 
teaching at our faculty. Students shared their experiences, indicated problems 
and gave suggestions on how to solve them. It appears that students have 
mostly experienced summative forms of evaluation such as student 
questionnaires at the end of every academic year. A student said: “Some 
professors ask us to write down our impressions about the course, and the 
things that we would like to change, but I think it is just a formality.” Students 
said that evaluation which was initiated and carried out by professors 
themselves was done only at Pedagogy department. Students are aware that 
evaluations are rare at the faculty, and that it is not easy to motivate academics 
to conduct evaluation. However, instead of having practical solutions for the 
joint initiative of academics and students to start evaluating teaching, they have 
shifted their attention to undefined and non-existing services that should 
supposedly take care of it: “I think that there is no such thing as evaluation at 

5 Namely, Thursday is the day for late nights out for students in Osijek. Since this course Methods of 
Upbringing is organized on Friday mornings, some students tend to be tired in classes. Professor 
recognized this as a problem very early on, but he only later found out from student M. Bungic 
what the reason was. Also, many students referred to this problem many times during evaluation. 
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this faculty. Therefore services should be formed that would be in charge of 
it.” Besides, there were ideas to reward good teachers on the one hand and to 
introduce some repressive methods on the other, in order to regulate 
academics’ behaviour; “If money makes the world go round, then it might be a 
good incentive to give money rewards to professors who have been graded best 
by students.” A student disagreed and said: “Why would they be given an 
additional reward? Those who do not perform their job well, they should be 
sanctioned, fired and that’s it.” Students agreed that their success at any course 
is partly a reflection of that academic’s teaching practices. A student suggested 
that professors cooperated with each other by means of critical friendship. 

Based on this discussion we may conclude that students, although they 
are not satisfied with evaluation at this point and its results, do not have a clear 
vision of how this problem might be solved. There have been just a few 
practical ideas which can contribute to its solution. This action research in 
which all the participants were both teachers and learners is an example of 
such a process. It is encouraging to see that students are aware of the 
importance of cooperation of all the participants in the teaching process. 
Encouraging is also the fact that they are ready to contribute to teaching 
improvement. We believe that students can be a source of creative ideas, but 
they should be given a chance to express these ideas, which is primarily an 
academic’s duty. 

The final activity in the workshop prompted students to create and 
conduct various evaluation procedures to evaluate the course Methods of 
Upbringing. A part of students designed and carried out a questionnaire. It 
consisted of five questions. It was copied and distributed to other students. The 
results of the questionnaire showed that students preferred classes that they 
designed and conducted (79%). Teacher’s activity was mostly graded as 
excellent (75%), and their class activity as good (62%). Some of the ideas for 
course improvement were: “To have classes at some other time, not so early 
while everybody is still sleeping”, “To encourage workshops conducted by 
students, this has been very positive so far”, “To take into consideration 
students’ mood and abilities before any activity is started”. 

A part of students designed an evaluation game, and a part of them 
designed an evaluation sheet. The evaluation sheet was creative and consisted 
of pictures which represented the level of course satisfaction and space for 
written description. One group designed and conducted an interview with two 
students and Professor Bognar. They asked questions about course satisfaction, 
their activity and suggestions for teaching improvement. Students recorded the 
interviews with their mobile phones. Teacher said in the interview and later 
during group presentations that he was very satisfied with the fact that students 
honestly expressed their opinion on the course Methods of Upbringing and 
other relevant matters. Open discussion and students’ answers in the interview 
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speak in favour of that fact: “There are no wrong answers. Your opinion is 
appreciated...”, “When a professor asks a question, it is very important to feel 
that you have enough space... that there is no pressure of thinking whether 
your words are important, correct. So, the space is important for people to feel 
free while talking.” We believe that the feeling of liberty on behalf of all the 
participants in teaching process is crucial for the exertion of substantial 
changes. However, we would agree with Hegel (1966) that “freedom as the 
ideal condition of what is as yet purely immediate and natural does not itself 
possess an immediate and natural existence. It still has to be earned through the 
endless mediation of discipline acting upon the powers of cognition and will”. 
In other words, freedom is not a gift, nor a natural condition; it must be fought 
for by learning and creative (co)work of the participants of teaching process. 

A few students observed and made notes on the process of designing 
and conducting evaluation, and one group had a task to watch the shortened 
version of the first workshop. By being able to comment on teaching practices 
they were given an opportunity to become critical friends. Students however 
made only positive comments. After the workshop the teacher asked: “Were 
there any problems?”, a group representative said: “Well, no. We have really 
tried to find some negative aspects, but there were none. We think everything 
was alright.” This statement is in contrast with the number of issues that were 
stated in chapter Evaluation of the workshop “What is a man?’’ Based on this 
problem we might conclude that teachers should not only regard students’ 
opinion when evaluating, but should also discuss their teaching practices with 
their colleagues, especially those who are acquainted with and apply modem 
teaching methods in their teaching organization. We also consider it important 
to teach students of teacher training studies to take a role of a critical friend, 
which is an important professional role that will help them be oriented towards 
change. 

For the evaluation of workshop an evaluation sheet was used with 
possibility to circle “like” if they liked the workshop of “dislike” if not. Most 
of the students (86%) liked the workshop, and some of them said: “I like 
creativity, positive outcomes and new insights”, “I like the overall atmosphere 
which was relaxed”, “I like creativity and organized approach”. 

As student M. Bungic organized this workshop, she had a role of a 
critical friend, but she was also in a position to feel what it is like to be in 
charge of teaching. After the class we talked to each other and the critical 
friend Ruzica Pazin Ilakovac about our impressions. She referred to students’ 
activity as a proof of the workshop success: 

We witnessed how they became more and more active and cheerful, and 
in the end, when they were given a self-guided task to come up with evaluation 
methods, it was obvious that they understood the subject very well and that 
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they were extremely successful at designing those small evaluation 
instruments. (R. Pazin Ilakovac, personal communication, June 3, 201 1) 


3. Instead of a conclusion 

We believe that all the criteria for evaluation of goals have been met. 
Formative evaluation has been conducted using various evaluation procedures 
such as evaluation sheets, questionnaires, interviews, critical friendship, etc. 
Based on the evaluation results we have tried to continuously improve teaching 
quality. Summative evaluation has been conducted using various evaluation 
procedures designed by students at the final workshop on evaluation of higher 
education. Based on the data presented, the conclusion can be reached that all 
the characteristics of fourth generation evaluation (Guba & Lincoln 1989, p. 
253-256) were present: It was discovered that evaluation is a social and 
political process during which students and teachers agree on the changes 
which should be in accord with their needs and interests. To do that, it is 
required that all the participants in the teaching process learn from one 
another. Dealing with the issues of evaluation of teaching we found out that it 
was a continuous process which demands constant critical re-examination of 
practice and finding creative solutions which may be appropriate for particular 
situations, but should not be regarded as universal “truths” applicable to all 
teaching situations. 

It has been discovered that evaluation is a developing process which is 
not predictable because every new step is based on the results of previous 
activities. This means that it is possible to specify methodological design, 
research problems and procedures at the end of evaluation procedure. 
Evaluation cannot be reduced to collecting and analysing data; it is a creative 
process during which all the interested participants take responsibility for the 
quality of teaching process. 

Action research usually does not result in conclusions but ideas for 
future projects which will enable the continuation of the process of change 
(Winter & Munn-Giddings 2001, p. 244). In our case, the results of action 
research, whose function was to improve the quality of higher education 
evaluation of an academic’s teaching practices, was presented at the expert 
conference of assistants and mentors at our faculty. Workshop participants 
were very positive about our experiences, which is obvious in some of the 
comments from the workshop evaluation sheet: 

(I liked) the cooperation with the student and her active role in the 
workshop; openness and readiness of workshop participants to cooperate and 
actively think about the use of a critical friend in higher education. 
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I would like to praise the very idea and this fresh impetus for all of us to 
try and improve our teaching processes to our own benefit and to our students’ 
benefit. Well done! (workshop participants, personal communication, February 
24, 2012). 

The idea of critical friendship has been incorporated in the Strategy for 
the development of the Faculty of Philosophy from 2011 to 2015. At April 
2012 meeting of head of departments it was agreed to form groups of critical 
friends among department members and across different departments, who 
would discuss possibilities of teaching improvement. This is a proof that our 
action research has contributed to the changes in the professional culture of our 
institution. In social context this means that, if there is a favourable atmosphere 
which accepts positive examples of change, action research experiences may 
become a part of the culture of close and wide community. Action research 
results are not generalized on a theoretical, but practical level - being an 
example and an impetus for the creative actions of other practitioners. 
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Mogucnosti evaluacije visokoskolske nastave 

Sazetak: Ovo akcijsko istrazivanje ostvareno je u okviru nastave kolegija Metodika 
odgoja tijekom jednoga semestra. Namjera nam je bila unaprijediti evaluaciju 
visokoskolske nastave. Kako bi to ostvarili koristili smo se razlicitim oblicima 
sumativne i formativne evaluacije, a poseban naglasak postavili smo na kriticko 
prijateljstvo izmedu studenta i profesora (suautora ovog rada). Za ostvarivanje 
kritickoga prijateljstva koristili smo se videozapisima nastave. 

Na kraju nastave studentima su bili ponudeni razliciti anketni upitnici i 
evaluacijski listici, a proveli smo i otvoreni standardizirani intervju sa skupinom 
studenata te radionicu koja je bila posvecena evaluaciji visokoskolske nastave. 

Nase istrazivanje pokazalo je kako studenti kao subjekti nastavnoga procesa 
mogu aktivno sudjelovati u ostvarivanju evaluacije, a njihova zapazanja i prijedlozi 
mogu potaknuti nastavnika na unaprjedenje svih etapa nastavnoga procesa, pa tako i 
evaluacije. Smatramo kako je evaluaciju u visokoskolskoj nastavi moguce 
unaprijediti edukacijom profesora i studenata o njezinoj vaznosti za kvalitetu 
nastavnoga procesa, upoznavanjem s razlicitim mogucnostima ukljucivanja studenata 
u taj proces, sto je ostvareno na jednom od struenih skupova koji je odrzan za 
nastavnike na Filozofskom fakultetu u Osijeku. 

Kljucne rijeci: evaluaeija, Cetvrta generaeija evaluacije, visokoskolska nastava, 
kritiCko prijateljstvo, akcijsko istrazivanje. 
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Evaluation im Hochschulunterricht 

Zusammenfassung: Diese Aktionsforschung wurde im Rahmen der 

Lehrveranstaltung Erziehungsmethodik wahrend eines Semesters realisiert. Wir 
wollten die Evaluation des Hochschulunterrichts verbessem. Um dies zu erreichen, 
haben wir verschiedene Formen der summativen und formativen Evaluation 
verwendet, mit einem besonderen Augenmerk auf die kritische Freundschaft 
zwischen dem Studenten und dem Professor (Co-Autor dieses Beitrages). Fur die 
Realisation der kritischen Freundschaft benutzten wir die Videoaufnahmen des 
Unterrichts. 

Am Unterrichtsende wurden den Studenten verschiedene Fragebogen und 
Evaluationspapiere angeboten, und wir ftihrten auch ein offenes standardisiertes 
Interview mit einer Studentengruppe, sowie einen Workshop, der sich mit der 
Evaluation des Hochschulunterrichts befasste. 

Unsere Forschung hat gezeigt, dass die Studenten als Subjekte des 
Unterrichtsprozesses aktiv an der Verwirklichung der Evaluation teilnehmen konnen, 
und ihre Bemerkungen und Anregungen konnen den Lehrer zur Verbesserung aller 
Etappen des Lehrprozesses bewegen, einschliefllich der Evaluation. Wir sind der 
Auffassung, dass die Evaluation im Hochschulunterricht verbessert werden kann, 
indem Lehrer und Studenten ttber ihre Bedeutung fur die Qua] i tat des Lehrprozesses 
geschult werden und die verschiedenen Moglichkeiten der Einbeziehung von 
Studenten in diesen Prozess kennen lemen. Dies wurde bei einer Fachtagung fur 
Lehrer realisiert, die an der Philosophischen Fakultat in Osijek stattgefunden hat. 

Schliisselbegriffe: Evaluation, vierte Generation der Evaluation, 

Hochschulunterricht, kritische Freundschaft, Aktionsforschung. 
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